Image Retrieval Supports Multimedia Authoring

Cognitive research indicates that multimedia presentations have benefits over text-only presentations, such as improved comprehension and recall by readers. In this paper, a method is proposed to enhance a text presentation with pictures, without using abstract semantic representations of either the text or the picture. The method is based on measuring similarity between the input text and text associated with the picture. If the texts are sufficiently similar, the picture may be included. Enriching text in this way may be useful for many purposes, such as multimedia authoring or automated presentation systems which do not have access to knowledge-rich sources. The proposed method will be implemented in a multimodal question answering system.